Fixed and Random Effects Models
How should we pool effect sizes in meta-analysis?
2024-07-30
Meta-analysis Recap
Meta-analysis Research
To estimate the true effect size from multiple studies:
- define research question (PICOT)
- collect results from appropriate studies
- apply statistical model*
Meta-analysis Data
- unit of observation is the study; dataset is \(K\) studies
- variables:
- estimated effect size (\(\widehat{\theta}_{k}\))
- standard error (\(\epsilon_{k}\))
- sample size (\(N_{k}\))
How should we pool effect sizes?
- How do we estimate what the true effect is?
- How should we weigh estimates?
Meta-analysis Models
- Fixed effects and random effects models make different assumptions about how data are generated.
- The chosen model should be appropriate for the underlying processes
Fixed Effects Model
Fixed Effects Assumption
- One true effect size (\(\theta_{\ F }\))
- Deviation from true effect is due to sampling error only
- One true effect size that exists for the population
- Errors are just due to random sampling (noise).
Fixed Effects Model
- Sampling error \(\epsilon_{\ k }\) is represented by the std. error \(SE_{k}\)
- SE is inversely related to sample size \(N_{k}\).
- \(\widehat{\theta}_{k}\) with small SE should be closer to true \(\theta\)
- These \(\widehat{\theta}_{k}\) should be given more weight
Inverse-Variance Weighting
Study weight \(w_{k}\) is inversely related to \(SE^2\)
\[ w_{k}=\frac{1}{SE^{2}_{k}} \]
The pooled estimate \(\widehat{\theta}\) is the weighted average:
\[ \widehat{\theta} = \frac{\Sigma_{k=1}^{K} \widehat{\theta}_{k} w_{k}}{\Sigma_{k=1}^{K} w_{k}} \]
Fixed Effects Model Interpretation
- Assumes all studies are sampling a completely homogenous population and measurement is performed in exactly the same manner.
- Model doesn’t allow between-study heterogeneity
- Too simplistic to account for real differences between studies in the underlying true effect.
Random Effects Model
Random Effects Assumption
- There are are universe of possible studies,
- Each study is different with it’s own true effect size
- Studies are randomly drawn from a distribution with mean true effect size \(\mu\)
Random Effects Model
A study’s effect size is an estimate of the true study effect size plus sampling error \[ \widehat{\theta}_{k}=\theta_{k} +\epsilon_{k} \]
The true effect size of study k is drawn from a distribution of effect sizes with mean \(\mu\), with error \(\zeta_{k}\).
\[ {\theta}_{k}=\mu+\zeta_{k} \]
Random Effects Model
- REM as a hierarchical structure
REM weighting
- we must first establish the variance of the distribution of true effect sizes, \(\tau^2\)
- this value informs the between-study heterogeneity for weighting studies
\[ w^{*}_{k}=\frac{1}{s^{2}_{k} + \tau^2} \]
And then we estimate the overall mean effect size using the same weighted average as before.
Estimating \(\tau^2\)
- Many procedures exist, all of which are far too complicated for this talk!
- The DerSimonian and Laird method is one of the oldest and very common
Exchangeability Assumption
- \(\zeta_{k}\) is independent of \(k\)
- there is no way to know what \(\zeta_{k}\) is a priori, ie. without looking at the data
- we say that different studies are assumed to be ‘exchangeable’ because we have no way to know \(\zeta_{k}\)
- This assumption may not be true…
Should we just always use the REM?
Only choose the FEM if you have clear reason to (rare)
For most scenarios, use the REM.
A Test for Between-Study Heterogeneity
- does the confidence interval for \(\tau^2\) include zero or not?
- if not, using REM is justified